195 research outputs found
Symblicit algorithms for optimal strategy synthesis in monotonic Markov decision processes
When treating Markov decision processes (MDPs) with large state spaces, using
explicit representations quickly becomes unfeasible. Lately, Wimmer et al. have
proposed a so-called symblicit algorithm for the synthesis of optimal
strategies in MDPs, in the quantitative setting of expected mean-payoff. This
algorithm, based on the strategy iteration algorithm of Howard and Veinott,
efficiently combines symbolic and explicit data structures, and uses binary
decision diagrams as symbolic representation. The aim of this paper is to show
that the new data structure of pseudo-antichains (an extension of antichains)
provides another interesting alternative, especially for the class of monotonic
MDPs. We design efficient pseudo-antichain based symblicit algorithms (with
open source implementations) for two quantitative settings: the expected
mean-payoff and the stochastic shortest path. For two practical applications
coming from automated planning and LTL synthesis, we report promising
experimental results w.r.t. both the run time and the memory consumption.Comment: In Proceedings SYNT 2014, arXiv:1407.493
Expectations or Guarantees? I Want It All! A crossroad between games and MDPs
When reasoning about the strategic capabilities of an agent, it is important
to consider the nature of its adversaries. In the particular context of
controller synthesis for quantitative specifications, the usual problem is to
devise a strategy for a reactive system which yields some desired performance,
taking into account the possible impact of the environment of the system. There
are at least two ways to look at this environment. In the classical analysis of
two-player quantitative games, the environment is purely antagonistic and the
problem is to provide strict performance guarantees. In Markov decision
processes, the environment is seen as purely stochastic: the aim is then to
optimize the expected payoff, with no guarantee on individual outcomes.
In this expository work, we report on recent results introducing the beyond
worst-case synthesis problem, which is to construct strategies that guarantee
some quantitative requirement in the worst-case while providing an higher
expected value against a particular stochastic model of the environment given
as input. This problem is relevant to produce system controllers that provide
nice expected performance in the everyday situation while ensuring a strict
(but relaxed) performance threshold even in the event of very bad (while
unlikely) circumstances. It has been studied for both the mean-payoff and the
shortest path quantitative measures.Comment: In Proceedings SR 2014, arXiv:1404.041
Words derivated from Sturmian words
AbstractA return word of a factor of a Sturmian word starts at an occurrence of that factor and ends exactly before its next occurrence. Derivated words encode the unique decomposition of a word in terms of return words. Vuillon has proved that each factor of a Sturmian word has exactly two return words. We determine these two return words, as well as their first occurrence, for the prefixes of characteristic Sturmian words. We then characterize words derivated from a characteristic Sturmian word and give their precise form. Finally, we apply our results to obtain a new proof of the characterization of characteristic Sturmian words which are fixed points of morphisms
- …